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Abstract 

Objectives:  This  study  examined  whether  clinical  evidence  in  medical  records 
confirms  ICD-9-CM  diagnoses  that  identify  the  Patient  Safety  Indicator  (PSI) 
accidental  puncture  or  laceration  (APL),  and  examined  the  utility  of  procedure 
codes  in  identifying  patients  who  suffered  an  APL.  Methods:  Validation  results 
were  based  on  a  retrospective  medical  record  review  of  67  inpatients  from  the 
year  2000  in  New  York  State  with  ICD-9-CM  codes  corresponding  to  APL. 
Secondary  procedure  codes  of  New  York  inpatients  during  2001  were  examined 
to  identify  patients  suffering  punctures  and  lacerations  who  were  not  captured  by 
the  PSI.  Results:  Clinical  conditions  represented  by  ICD-9-CM  codes  used  in  the 
PSI  were  frequently  not  present  in  the  records  of  the  67  patients.  Cases  of  APL 
that  were  missed  by  the  PSI  were  identified  using  procedure  codes.  Conclusions: 
Not  all  of  the  patients  identified  by  the  PSI  meet  the  intent  of  the  indicator.  The 
PSI  also  misses  cases  of  APL  by  failing  to  use  procedure  codes  in  the  algorithm. 


Introduction 

Various  types  of  algorithms  that  identify  complications  of  care  and  adverse 
events  in  hospitalized  patients  have  been  developed  using  administrative  data. 
These  efforts  have  included  algorithms  identifying  specific  procedures  or 
conditions1  9  and  algorithms  that  identify  broader  groups  of  patients.10  17  The 
Complications  Screening  Program  (CSP)  by  Iezzoni  et  al.11  and  the  Patient  Safety 
Indicators  (PSIs)  by  Miller  et  al.  have  significantly  contributed  to  the  most 
recent  set  of  inpatient  quality  indicators,  developed  by  Stanford  University  and 
the  University  of  California  with  a  grant  from  the  Agency  for  Healthcare 
Research  and  Quality  (AHRQ).19  These  new  AHRQ  Quality  Indicators  identify  a 
number  of  complications  of  care  or  adverse  events  and  have  been  made  available 
to  researchers.20 

There  has  been  limited  validation  of  administrative  data  algorithms.  The  CSP 
algorithms  were  evaluated  by  Iezzoni  and  colleagues  while  the  AHRQ  PSIs 
were  evaluated  as  part  of  the  development  process.19  Several  issues  were  raised 
by  the  AHRQ  physician  consultants  during  the  development  of  the  accidental 
puncture  or  laceration  PSI,  including  the  contribution  of  trauma  patients  to  events 
identified  by  the  PSI.  A  second  concern  was  that,  while  the  intent  of  the  APL  PSI 
is  to  identify  all  punctures  and  lacerations  that  occur  in  the  inpatient  setting,  only 
the  most  serious  adverse  events  would  be  captured  with  the  diagnosis  codes 
selected.  Researchers  involved  in  the  development  of  the  PSI  measures  and  the 
AHRQ  have  called  for  additional  review  and  validation  of  these  indicators19'25  as 
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researchers  have  begun  to  apply  the  indicators  to  their  work.26  28  In  this  paper,  we 
present  findings  of  our  analysis  and  validation  of  the  indicator  intended  to  identify 
cases  of  accidental  puncture  or  laceration  (APL)  that  occur  in  the  hospital  after  a 
procedure. 

Specifically,  the  purposes  of  the  study  are  to  use  medical  record  reviews  to  (1) 
validate  the  occurrence  of  in-hospital  APLs  associated  with  the  PSI  for  APL,  and 
(2)  investigate  the  occurrence  of  in-hospital  APLs  associated  with  minor 
lacerations  or  punctures.  We  also  examine  procedure  codes  of  cases  captured  by 
the  PSI  to  determine  if  any  changes  to  the  algorithm  can  be  made  to  identify 
additional  events.  The  use  of  procedure  codes  used  to  identify  events  are  similar 
to  the  methodology  used  by  Iezzoni  for  CSP  17,  procedure-related  perforation  or 
laceration,11  and  the  AHRQ  experimental  PSI,  suture  of  laceration.19 


Methods 

Data  source 

The  AHRQ  PSI  program,  Version  2.1,  Revision  1,  May  2003, 20  contains 
modules  that  define  denominator  and  numerator  events,  calculate  overall  observed 
rates,  and  calculate  risk-adjusted  rates  of  patient  safety  events.  These  programs 
were  applied  to  data  from  the  New  York  State  inpatient  hospitalization 
administrative  data  system,  the  Statewide  Planning  and  Research  Cooperative 
System  (SPARCS),  for  discharges  in  2001. 

Case  identification 

Measures  of  Patient  Safety  Based  on  Hospital  Administrative  Data — The 
Patient  Safety  Indicators,  Technical  Review  5, 19  defines  the  denominator  for  the 
APL  indicator  as  all  medical  and  surgical  discharges  defined  by  specific  diagnosis 
related  groups  (DRGs),  excluding  obstetrical  patients  in  medical  diagnostic 
category  (MDC)  14.  Numerator  events  are  defined  as  medical  and  surgical 
discharges  with  International  Classification  of  Diseases,  Ninth  Revision,  Clinical 
Modification  (ICD-9-CM)  codes"  for  accidental  puncture  or  laceration  in  any 
secondary  diagnosis  field. 

Data  analysis 

The  validation  sample  is  based  on  the  New  York  State  inpatient  adverse  event 
report  system,  New  York  Patient  Occurrence  Reporting  and  Tracking  System 
(NYPORTS),  which  captures  a  variety  of  adverse  events.  Rates  of  inpatient 
hospital  adverse  event  reporting  were  calculated,  where  the  total  number  of  events 
reported  to  NYPORTS  was  in  the  numerator,  and  the  total  number  of  inpatient 
discharges  was  in  the  denominator.  New  York  State  hospitals  were  ranked  by 
NYPORTS  reporting  rate,  with  a  sample  of  high,  moderate,  and  low  reporting 
hospitals  selected.  Records  of  inpatients  admitted  with  APL  as  a  secondary 
diagnosis  in  2000  were  extracted  from  SPARCS  for  these  sample  hospitals.  While 
the  APL  PSI  includes  cardiac  patients,  our  validation  sample  did  not  include 
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cardiac  patients.  New  York  State  has  several  adverse  event  reporting  systems, 
including  the  New  York  State  Adult  Cardiac  Reporting  System  and  NYPORTS. 
The  validation  sample  used  for  this  analysis  was  selected  for  inpatient  adverse 
events  which  are  reportable  to  NYPORTS  and  excludes  cardiac  patients  because 
these  events  are  reportable  to  New  York  State  Adult  Cardiac  Reporting  System. 

Records  for  67  patients  who  were  identified  using  the  AHRQ  PSI  definition 
for  APL  were  provided  to  an  independent  review  agent,  the  Island  Peer  Review 
Organization  (IPRO),  for  medical  chart  review.  IPRO  used  registered  nurses  to 
conduct  retrospective  medical  record  reviews  using  a  standard  validation  review 
instrument  to  determine  if  a  reportable  event  occurred.  The  validation  review 
instrument  was  used  to  collect  infonnation  corroborating  the  diagnosis  of  APL, 
confirming  the  timing  of  the  event  (present  at  admission  or  developed  in  the 
hospital),  confirming  the  severity  of  the  event  (penetration  of  lumen  or  removal  of 
organ  versus  laceration  or  penetration  of  serosal  or  muscular  layers),  and 
ascertaining  if  the  patient  suffered  trauma.  The  validation  review  summaries  were 
analyzed  to  detennine  if  the  potential  APL,  as  defined  using  the  AHRQ  APL 
definition,  was  an  event  that  occurred  during  the  index  hospitalization. 

Primary  and  secondary  procedure  codes  of  cases  captured  by  the  PSI  were 
also  examined  to  detennine  if  accidental  puncture  or  laceration  patients  with 
particular  primary  procedures  could  be  identified  using  specific  secondary 
procedure  codes. 

Analytic  approach 

Numerator  events  were  validated  using  medical  record  reviews.  The 
proportion  of  cases  confinned  as  in-hospital  APL  events,  the  proportion  of  cases 
not  confirmed  due  to  the  timing  of  the  diagnosis,  the  proportion  of  cases  not 
confirmed  due  to  lack  of  clinical  evidence  in  the  medical  record,  and  the 
proportion  of  trauma  cases  were  calculated.  In  addition,  the  proportions  of 
confirmed  cases  with  a  minor  puncture  or  laceration  were  identified. 

The  number  of  numerator  and  denominator  events  identified  with  the  PSI  and 
the  rate  of  APL  events  were  calculated.  Numerator  and  denominator  counts  and 
the  APL  rate  were  also  calculated  after  patients  who  were  identified  as  being 
erroneously  captured  by  the  PSI  were  removed.  The  number  of  patients  with 
secondary  procedure  codes  indicating  puncture  or  laceration  for  specific  primary 
procedures  who  were  not  captured  by  the  PSI  was  also  calculated. 


Results 

Using  the  original  algorithm  for  the  PSI  and  applying  it  to  2001  inpatient 
discharges  in  New  York  State,  we  identified  4,643  numerator  events  and 
1,947,441  patients  in  the  denominator,  which  is  2.4  accidental  lacerations  or 
punctures  per  1,000  discharges  (Table  1). 

The  PSI  erroneously  identifies  cases  that  have  accidental  puncture  or 
laceration  diagnosis  codes  in  the  principal  diagnosis  position.  This  occurs  when  a 
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patient  has  a  primary  diagnosis  of  accidental  puncture  or  laceration  and  a 
secondary  diagnosis  of  accidental  puncture  or  laceration  in  the  same  record. 
Because  the  intent  of  the  indicator  is  to  identify  cases  that  occur  after  admission, 


Table  1.  Accidental  puncture  or  laceration  (APL)  events  based  on  revised  exclusions, 
New  York  State,  2001 


Exclusions* 

No.  of  records  with 
APL 

Risk  pool  for  APLf 

APL  rate  per  1,000 
discharges 

PSI  algorithm 

4,643 

1,947,441 

2.4 

Eliminate  1st  DX=APL 

4,408 

1,946,862 

2.3 

*PSI  =  Patient  Safety  Indicators;  1st  =  principal;  DX  =  diagnosis. 
tRisk  pool  definition  based  on  PSI  inclusion  and  exclusion  algorithm. 


these  cases  were  eliminated  from  the  PSI.  The  revised  definition  yields  4,408 
events  from  1,946,862  at-risk  patients,  which  is  2.3  accidental  lacerations  or 
punctures  per  1,000  discharges.  This  rate  of  2.3  per  1,000  discharges  is  lower  than 
recently  published  estimates  using  the  AHRQ  PSIs.  Romano  et  al.  established  an 
accidental  puncture  and  laceration  rate  of  3.24  per  1,000  discharges  using  data 
from  the  2000  Nationwide  Inpatient  Sample." 

Table  2  indicates  that  for  the  67  cases  in  the  validation  dataset,  two  patients 
(3.0  percent)  did  not  meet  the  definition  of  an  APL  because  they  did  not  have 
clinical  evidence  in  their  charts  indicating  the  presence  of  an  accidental  puncture 
or  laceration;  and  two  patients  (3.0  percent)  did  not  meet  the  intent  of  the  APL 
PSI  definition  because  they  had  an  accidental  puncture  or  laceration  present  at 
admission.  The  medical  chart  review  also  revealed  three  trauma  cases 
(4.5  percent),  two  of  which  had  the  puncture  or  laceration  caused  by  the  trauma 
prior  to  admission  and  one  that  had  the  complication  after  admission.  Ten  (14.9 
percent)  of  the  cases  had  an  accidental  laceration  or  puncture  coded,  but  the 
review  indicated  that  the  injuries  were  not  serious.  These  complications  involved 
either  a  cut  or  tear  of  the  muscle  or  serosal  layers,  and  did  not  include  any 
penetration  of  lumen  or  removal  of  organs.  The  remaining  50  cases  (74.6  percent) 
appeared  to  be  serious  complications  of  care. 


Table  2.  Validation  of  APL  events,  New  York  State,  2000 


Data  characteristics 

N 

% 

Validation  set 

67 

- 

Non-APL  cases 

2 

3.0 

Serious  cases 

50 

74.6 

Not  serious  cases 

10 

14.9 

Trauma 

2 

3.0 

Trauma  and  PSI 

1 

1.5 

Readmits 

2 

3.0 
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Lacerations  and  punctures  are  captured  in  the  PSI  using  diagnosis  codes.  It  is 
also  possible  to  identify  lacerations  and  punctures  using  secondary  procedure 
codes.  (See  Table  3  for  secondary  procedure  codes  indicative  of  accidental 
puncture  and  injury  for  select  principal  procedures.) 


Table  3.  Secondary  diagnosis  or  procedure  codes  for  identifying  potential  accidental 
puncture  or  laceration  patients 


Injury  types 

ICD-9-CM  codes 

Bladder  injury 
associated  with 
hysterectomy 

57.81  -  Suture  of  laceration  of  bladder 

57.89  -  Other  repair  of  bladder:  bladder  suspension,  not  elsewhere 
classified;  cystopexy  NOS;  repair  of  old  obstetric  laceration  of  bladder 

Bowel  injury  associated 
with  cholecystectomy 

46.71  -  Suture  of  laceration  of  duodenum 

46.73  -  Suture  of  laceration  of  small  intestine,  except  duodenum 

46.75  -  Suture  of  laceration  of  large  intestine 

46.79  -  Other  repair  of  intestine 

Abdominal  injury 
associated  with  lysis  of 
peritoneal  adhesions 

569.83  -Perforation  of  intestine 

07.44  -  Repair  of  adrenal  gland 

39.3  -  Suture  of  vessel 

39.59  -  Other  repair  of  vessel 

41 .95  -  Repair  and  plastic  operations  on  spleen 

44.61  -  Suture  of  laceration  of  stomach 

44.69  -  Other  repair  of  stomach:  Inversion  of  gastric  diverticulum 

46.71  -  Suture  of  laceration  of  duodenum 

46.73  -  Suture  of  laceration  of  small  intestine,  except  duodenum 

46.75  -  Suture  of  laceration  of  large  intestine 

46.79  -  Other  repair  of  intestine 

48.71  -  Suture  of  laceration  of  rectum 

48.79  -  Other  repair  of  rectum:  repair  of  old  obstetric  laceration  of 
rectum;  Inversion  of  gastric  diverticulum 

50.6  -  Repair  of  liver 

51 .71  -  Simple  suture  of  common  bile  duct 

51 .79  -  Repair  of  other  bile  ducts:  closure  of  artificial  opening  of  bile 
duct  NOS;  Suture  of  bile  duct  NOS 

51 .91  -  Repair  of  laceration  of  gallbladder 

52.95  -  Other  repair  of  pancreas:  fistulectomy  of  pancreas;  simple 
suture  of  pancreas 

54.63  -  Other  suture  of  abdominal  wall:  suture  of  laceration  of 
abdominal  wall 

54.64  -  Suture  of  peritoneum:  secondary  suture  of  peritoneum 

54.72  -  Other  repair  of  abdominal  wall 

54.73  -  Other  repair  of  peritoneum:  suture  of  gastrocolic  ligament 
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Table  3.  Secondary  diagnosis  or  procedure  codes  for  identifying  potential  accidental 
puncture  or  laceration  patients,  cont. 


Injury  types 

ICD-9-CM  codes 

54.74  -  Other  repair  of  omentum:  epiplorrhaphy;  graft  of  omentum; 
omentopexy;  reduction  of  torsion  of  omentum 

54.75  -  Other  repair  of  mesentery:  mesenteric  placation; 
mesenteropexy 

55.81  -  Suture  of  laceration  of  kidney 

55.89  -  Other  repair  of  kidney 

56.82  -  Suture  of  laceration  of  ureter 

56.89  -  Other  repair  of  ureter:  graft  of  ureter;  replacement  of  ureter 
with  ileal  segment  implanted  into  bladder 

57.81  -  Suture  of  laceration  of  bladder 

57.89  -  Other  repair  of  bladder:  bladder  suspension,  not  elsewhere 
classified;  cystopexy  NOS;  repair  of  old  obstetric  laceration  of  bladder 

Abdominal  injury 
associated  with 
nephroureterectomy 

See  codes  for  abdominal  injury  associated  with  lysis  of  peritoneal 
adhesions 

Spinal  injury  associated 
with  lumbar  surgery 

03.59  -  Other  repair  and  plastic  operations  on  spinal  cord  structures: 
diastematomyelia;  spinal  bifida  NOS;  spinal  cord  NOS;  spinal 
meninges  NOS;  vertebral  arch  defect 

Note:  Five-digit  codes  are  diagnosis  codes;  all  others  are  procedure  codes. 


Table  4  presents  counts  of  events  that  appear  to  be  accidental  punctures  or 
lacerations  based  on  analysis  of  the  primary  and  secondary  procedure  codes  for 
patients  who  had  one  of  five  select  primary  procedures.  For  instance,  for  patients 
who  underwent  a  hysterectomy  as  the  principal  procedure,  the  table  shows  that 
194  patients  with  bladder  injuries  were  captured  by  the  PSI,  and  73  (27.3  percent) 
inpatients  had  these  injuries  during  2001  in  New  York  State  and  were  not 
captured  by  the  PSI.  Cholecystectomy  patients  who  suffered  bowel  injuries  were 
not  captured  by  the  PSI  20.8  percent  of  the  time.  For  patients  undergoing  lysis  of 
peritoneal  adhesions  who  suffered  abdominal  injuries,  47.4  percent  were  not 
captured  by  the  PSI.  A  total  of  53.8  percent  of  nephroureterectomy  patients 
developing  abdominal  injuries  were  not  captured  by  the  PSI,  and  19.6  percent  of 
lumbar  surgery  cases  with  spinal  injuries  were  not  identified  by  the  PSI. 
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Table  4.  Potential  laceration  or  puncture  adverse  events  associated  with  five  principal 
procedures 


Procedures 

Events  captured  with 

PSI  definition 

Events  not  captured 
with  PSI  definition 

N 

% 

N 

% 

Bladder  injuries  caused  by 
hysterectomy 

194 

72.7 

73 

27.3 

Bowel  injuries  caused  by 
cholecystectomy 

61 

79.2 

16 

20.8 

Adnominal  injuries  caused  by 
lysis  of  peritoneal  adhesions 

91 

52.6 

82 

47.4 

Abdominal  injuries  caused  by 
nephroureterectomy 

24 

46.2 

28 

53.8 

Spinal  injuries  caused  by  lumbar 
surgery 

74 

80.4 

39 

19.6 

Discussion 

With  the  publication  of  the  Institute  of  Medicine’s  recent  report,  To  Err  Is 
Human:  Building  a  Safer  Health  System,  there  has  been  an  emphasis  on 
identifying  adverse  events  that  occur  in  the  inpatient  setting  as  a  means  of 
informing  hospitals  of  the  extent  of  various  problems,  and  as  a  means  of 
measuring  the  impact  of  efforts  to  improve  quality.  This  is  a  difficult  undertaking 
because  it  is  expensive  to  create  databases  for  this  purpose  and  because  existing 
administrative  databases  that  were  created  for  other  purposes  are  limited  in  their 
ability  to  supply  this  information.  AHRQ  has  developed  Patient  Safety  Indicators 
to  be  used  with  administrative  data  based  on  the  pioneering  efforts  of  Iezzoni  and 
colleagues  as  well  as  work  generated  from  grants  to  Stanford  University  and  the 
University  of  California  at  San  Francisco. 

AHRQ  has  published  the  PSIs,  which  include  commentary  by  medical  and 
surgical  specialists  who  consulted  on  algorithm  development.19  Several  issues 
were  raised  by  the  panels  with  regard  to  the  accidental  puncture  or  laceration  PSI, 
including  the  contribution  of  trauma  patients  to  events  identified  by  the  PSI.  A 
second  concern  of  the  physician  reviewers  was  that,  while  the  intent  of  the  APL 
PSI  is  to  identify  all  punctures  and  lacerations  that  occur  in  the  inpatient  setting,  it 
is  possible  that  only  the  most  serious  adverse  events  would  be  captured  with  the 
diagnosis  codes  selected.  The  results  of  our  validation  sample  indicate  that  both  of 
these  issues  affect  the  PSI  results. 

The  validation  results  indicate  that  the  PSI  does  not  include  criteria  that  are 
explicit  enough  to  exclude  patients  with  a  primary  diagnosis  of  accidental 
puncture  or  laceration.  Two  cases  (3  percent)  in  the  validation  sample  had  a 
principal  diagnosis  of  accidental  puncture  or  laceration  and  were  readmissions  for 
a  puncture  or  laceration.  These  patients  do  not  meet  the  indicator’s  intent,  which 
is  to  identify  occurrence  of  an  accidental  puncture  or  laceration  during  the  index 
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admission.  Changing  the  algorithm  and  applying  it  to  New  York  State  discharges 
in  2001  yields  4,408  events  from  1,946,862  at-risk  patients,  which  is  2.3 
accidental  lacerations  or  punctures  per  1,000  discharges,  compared  to  the  2.4 
accidental  laceration  or  punctures  per  1,000  discharges  resulting  from  the  original 
algorithm  (Table  1).  This  deficiency  in  the  algorithm  could  be  addressed  by 
excluding  accidental  puncture  or  laceration  codes  from  the  primary  diagnosis 
field. 

A  second  finding  of  the  validation  sample  is  that  trauma  patients  are  being 
captured  by  the  PSI.  While  our  sample  of  trauma  cases  (three  cases,  4.5  percent) 
is  too  small  to  draw  definitive  conclusions,  it  is  a  concern  that  two  out  of  the  three 
trauma  cases  that  were  evaluated  were  not  accidental  punctures  or  lacerations;  the 
injuries  were  suffered  prior  to  hospitalization.  While  AHRQ  recognized  this  issue 
during  algorithm  development,  and  chose  to  include  these  patients,  the  benefit  of 
identifying  a  small  number  of  APLs  in  trauma  patients  may  be  outweighed  by 
including  events  in  the  numerator  that  are  not  actual  APLs.  If  trauma  patients  are 
going  to  be  identified  and  eliminated  from  the  PSI,  the  definition  of  a  trauma 
patient  specified  in  the  PSI  documentation  appears  to  be  too  broad  (a  single 
ICD-9-CM  code  from  selected  diagnoses  in  the  800-958  range  or  one  of  several 
DRGs).  A  more  restrictive  definition  that  requires  at  least  two  ICD-9-CM 
diagnosis  codes  in  the  800-956  range  or  the  principal  diagnosis  in  this  range 
would  be  more  appropriate  to  capture  true  trauma  cases.  Using  this  more 
restrictive  definition  of  trauma,  we  identified  128  cases  in  the  APL  PSI  for  New 
York  State  in  2001,  2.8  percent  of  the  total  events  identified  for  this  period. 

An  additional  concern  raised  by  the  medical  consultants  during  the 
development  of  the  PSI  was  that  the  algorithm  may  capture  only  the  most  serious 
adverse  events  and  that  moderately  injured  patients  or  patients  with  minor  injuries 
will  go  undetected.  This  is  a  concern  because  the  intent  of  the  PSI  is  to  capture  all 
events  occurring  in  the  inpatient  setting,  and  the  diagnosis  codes  used  may  only 
capture  the  most  serious  events.  If  quality  improvement  initiatives  are  to  be 
developed  based  on  events  captured  in  the  PSI,  surgeons  causing  minor  or 
moderate  injuries  may  not  be  identified  by  the  APL  PSI  and  will  not  be  targeted 
for  quality  improvement.  Using  a  simple  definition  of  severity  where  puncture  of 
a  lumen  or  removal  of  an  organ  represents  a  serious  event,  and  puncture  or 
laceration  of  the  muscle  or  serosal  layer  is  considered  a  minor  event,  the 
validation  sample  results  indicate  that  a  small  number  of  cases  (10  or  14.9 
percent)  are  not  serious  complications.  The  ascertainment  of  severity  was  based 
on  review  of  the  medical  chart  documentation,  and,  unfortunately,  it  does  not 
appear  that  serious  and  minor  events  can  be  differentiated  using  diagnosis  and 
procedure  codes  recorded  in  the  administrative  record.  Based  on  this  finding, 
patients  with  minor  injuries  captured  by  the  PSI  algorithm  cannot  be  easily 
identified  with  administrative  data,  but  users  of  the  PSI  should  be  aware  of  the 
proportion  of  cases  that  may  not  be  serious  events. 

The  validation  results  reveal  that  the  diagnosis  codes  used  to  identify 
punctures  and  lacerations  are  accurate,  with  97.0  percent  (65)  of  the  cases  having 
an  indication  of  an  injury  in  the  medical  chart,  but  not  all  of  these  cases  meet  the 
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intent  for  the  PSI.  A  small  proportion  of  the  events  (3.0  percent)  were 
readmissions  that  can  be  excluded  with  a  minor  change  to  the  algorithm.  The 
concerns  of  the  medical  consultants  involved  in  the  development  of  the  PSI — that 
only  the  most  serious  adverse  events  would  be  captured  with  the  diagnosis  codes 
used  in  the  PSI — appear  to  be  borne  out  by  this  analysis.  Only  14.9  percent  of  the 
events  did  not  appear  to  be  serious  injuries,  and  4.5  percent  involved  trauma 
cases.  It  is  unclear  whether  identification  of  all  punctures  and  lacerations  is 
necessary  to  facilitate  quality  improvements  in  the  inpatient  setting,  or  if  it  is 
sufficient  to  capture  the  most  serious  cases.  It  is  likely  that  many  of  the  trauma 
patients  identified  by  the  indicator  suffered  injuries  outside  of  the  control  of 
medical  personnel.  Eliminating  cases  that  did  not  have  an  indication  of  puncture 
or  laceration  in  their  chart  (3  percent),  readmissions  (3  percent),  and  trauma  cases 
(4.5  percent)  leaves  88.5  percent  of  the  cases  identified  by  the  PSI  as  cases  of 
injury  that  may  be  appropriate  for  hospitals  to  identify  and  examine  in  order  to 
make  quality-of-care  improvements. 

These  validation  results  are  generally  applicable  to  the  patients  identified  by 
the  APL  PSI,  except  for  cardiac  patients.  Our  sample  did  not  include  cardiac 
patients,  and  these  patients  make  up  approximately  23  percent  of  the  patients 
identified  by  the  PSI  in  New  York  State  during  2001.  Additional  validation 
studies  that  include  cardiac  patients  should  be  conducted  to  detennine  if  these 
results  are  applicable  to  cardiac  patients. 

The  APL  PSI  relies  on  a  single  diagnosis  code  and  10  external  cause-of-injury 
codes  (E-codes)  to  identity  adverse  events.  It  is  possible  to  identify  punctures  and 
lacerations  using  procedure  codes,  as  evidenced  by  Iezzoni’s  CSP  17,  procedure- 
related  perforation  or  laceration,11  and  the  AHRQ’s  experimental  PSI,  suture  of 
laceration.19  CSP  17  uses  four  diagnosis  codes  in  a  secondary  field  and  16 
procedure  codes  in  a  secondary  field  to  identify  patients.  The  AHRQ  experimental 
PSI  uses  only  procedure  codes  in  a  secondary  field  to  identify  patients.  The 
approach  taken  in  this  analysis  uses  procedure  codes  in  a  secondary  position,  but 
we  also  limited  the  patients  included  in  the  study  to  those  having  specific 
secondary  procedures  associated  with  certain  primary  procedures.  Table  3  lists  a 
proposed  set  of  secondary  procedure  codes  that  can  be  used  to  identity  puncture 
and  laceration  in  patients  undergoing  specific  primary  procedures. 

The  APL  PSI  captures  patients  who  have  many  different  primary  procedures. 
One  of  the  primary  procedures  that  occurs  often  in  this  PSI  is  hysterectomy. 
Hysterectomies  are  associated  with  certain  operative  complications,  ’  ‘  including 
accidental  laceration  of  other  organs,  particularly  the  bladder.  When  we  examined 
the  records  captured  by  this  PSI  using  New  York  State  discharge  data  from  2001 
and  selected  the  subset  with  hysterectomy  as  the  primary  procedure,  we  found 
472  hysterectomies  with  194  bladder  lacerations  (ICD-9-CM  procedure  codes 
57.81,  suture  of  laceration  of  bladder  or  57.89,  other  repair  of  bladder)  in  any 
secondary  procedure  field.  Clearly,  a  large  proportion  (41  percent)  of  the 
hysterectomies  was  captured  by  the  PSI  because  of  accidental  bladder  laceration, 
as  indicated  by  a  secondary  procedure.  To  determine  if  the  PSI  is  capturing  all  of 
the  bladder  lacerations  occurring  in  hysterectomies,  we  examined  how  many 
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records  with  a  primary  procedure  code  of  hysterectomy  had  a  secondary 
procedure  of  bladder  laceration  in  the  absence  of  the  PSI  definition  for  accidental 
puncture  or  laceration.  Using  data  from  New  York  State,  2001,  267  bladder 
lacerations  were  identified  in  patients  undergoing  hysterectomies,  and  73  of  the 
267  (27.3  percent)  were  not  captured  by  the  PSI.  For  this  primary  procedure,  the 
discharge  data  can  be  examined  to  identify  the  particular  sites  of  accidental 
puncture  or  lacerations  by  reviewing  secondary  procedure  fields.  This  allows  us  to 
estimate  the  completeness  of  the  indicator  for  this  particular  primary  procedure. 

This  method  of  identifying  APLs  may  not  accurately  identify  accidental 
punctures  or  procedures  in  all  cases,  because  it  may  be  possible  that  a  patient  has 
comorbid  conditions  that  require  additional  secondary  procedures,  in  addition  to 
the  principal  procedure,  as  part  of  patient  treatment.  Our  intent  is  not  to 
definitively  identify  secondary  procedure  codes  that  indicate  APLs,  but  rather  to 
illustrate  the  potential  utility  of  this  approach  to  expanding  the  definition  of  the 
APL  PSI.  To  ascertain  which  patients  may  be  identified  using  procedure  codes  in 
addition  to  hysterectomies,  we  first  examined  the  distribution  of  patients  by 
primary  procedure  in  the  APL  PSI.  Several  primary  procedures  occurred  at  a  high 
frequency  and  were  used  for  this  analysis. 

For  patients  undergoing  five  principal  procedures,  Table  4  presents  the 
proportion  of  cases  not  captured  by  the  PSI.  While  it  is  impressive  that  bladder 
injuries  from  hysterectomies,  bowel  injuries  from  cholecystectomies,  and  spinal 
injuries  from  lumbar  surgery  are  largely  captured  by  the  PSI,  19-27  percent  of  the 
events  from  these  three  primary  procedures  go  undetected.  Abdominal  injuries 
due  to  lysis  of  peritoneal  adhesions  and  nephroureterectomy  are  not  captured  well 
by  the  PSI,  with  approximately  50  percent  of  these  events  not  identified. 
Examining  five  primary  procedures,  238  cases  that  appear  to  be  APLs  but  were 
not  captured  by  the  PSI  were  identified.  The  number  of  events  captured  by  the 
PSI  could  be  increased  by  approximately  6  percent  if  these  events  were  added, 
and  potentially  more  if  additional  primary  procedures  were  analyzed. 

Several  of  the  PSIs  are  based  on  capturing  E-code  data,  including  the  APL 
PSI.  These  codes  are  used  to  classify  environmental  events,  circumstances,  and 
other  conditions  as  the  cause  of  injury  and  other  adverse  effects.  E-codes  are 
critical  for  capturing  information  on  the  mechanism  (e.g.,  motor  vehicle  traffic, 
fall,  fire/burn,  firearm,  poisoning)  and  intent  (e.g.,  unintentional,  assault,  suicide 
attempt)  of  injuries  at  the  State  and  local  levels.  However  not  all  States  have  been 
able  to  achieve  a  high  level  of  E-coding  in  statewide  hospital-based  data  systems, 
nor  do  all  States’  unifonn  billing  committees  require  use  of  E-codes.  Because  of 
these  limitations,  it  is  especially  important  to  explore  alternative  methods  of 
identifying  adverse  events  that  do  not  rely  on  E-codes.  The  use  of  CSP  17, 
procedure-related  perforation  or  laceration,11  the  AHRQ’s  experimental  PSI, 
suture  of  laceration,19  or  the  use  of  specific  secondary  procedure  codes  for  select 
primary  procedures  are  approaches  to  capturing  these  important  adverse  events 
without  relying  on  the  E-codes. 
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Conclusions 

The  AHRQ  PSIs  are  based  on  the  best  available  evidence  regarding  the  use  of 
administrative  data  to  identify  adverse  events.  AHRQ’s  work  has  improved  the 
quality  of  the  indicators,  but  any  indicator  based  on  secondary  data  will  be  subject 
to  limitations.  In  this  analysis,  the  definition  of  a  particular  PSI  has  been  closely 
examined  and  several  methods  of  improving  the  indicator  have  been  identified. 
The  revised  indicator  can  be  used  on  the  same  datasets  that  are  appropriate  for  the 
original  indicator.  Our  intent  was  to  recommend  changes  that  could  be  easily 
implemented  without  requiring  any  additional  data  to  the  standard  inpatient 
discharge  dataset.  We  look  forward  to  future  efforts  to  use  administrative  data  to 
more  accurately  and  completely  ascertain  the  prevalence  of  iatrogenic  adverse 
events. 
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